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education by making Smithsonian experts and collections accessible to everyone regardless of where they live. 
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Background 


Since 2011, the Smithsonian Center for Learning and Digital Access (GCLDA) has strived to 
better understand and address the needs of educators utilizing digital assets through a variety 
of research and user testing studies that have led to the creation of a new digital learning 
platform, the Smithsonian Learning Lab (SLL)'. The Smithsonian Learning Lab provides access 
to the digital resources from across the Smithsonian’s 19 museums, 9 major research centers, 
and the National Zoo, to be used as real-world learning experiences. With a repository of over 
1.6 million objects and a new resource being digitized and added every 6 seconds, the 
Learning Lab provides specialized tools to aid in the discovery and creative use of its rich 
digital materials. For students using the Learning Lab, it is designed to aid in building lasting 
knowledge and critical skills that take learners from simply finding resources to thoughtful 
selection, examination, organization, and creation of new resources. 


The SLL, as it currently exists, was largely informed by the input and practice of diverse and 
effective educators.’ Therefore the goal of this Usage Data Analysis, as a piece of a larger 
research effort, Understanding the Needs of Student Users of Digital Smithsonian Resources, 
focuses on comparing activities performed on the SLL by users in the project's target age 
range (13-17) to adult users. While not intended to be a mere validation of SLL’s features, the 
design for this review and summary report is to lend additional insight into how digital systems, 
tools, pedagogy and content, can be adapted to better meet students’ learning needs. As 
educational psychologist Paul A. Kirschner> points out, “If the student is viewed as the end 
user... participatory design needs to include a more direct participation/contribution of the 
student in the design of (technology enhanced) learning environments”. The overall project will 
assimilate the findings of this Usage Analysis along with other research to address some key 
questions around methods or requirements for enhancing student motivation and engagement 
with digital content and tools. 


1. What are the ways that students engage with digital content in academic settings? 

2. What are the motivations for student use of digital content? 

3. What are the interface requirements/scaffolds needed to enable and enhance student 
engagement with rich digital resources? 


' Milligan, D., and M. Wadman, M. 2015. "From Physical to Digital: Recent Research into the Discovery, 
Analysis . . .” 
<http://mw2015.museumsandtheweb.com/paper/from-physical-to-digital-recent-research-into-the-disc 


overy-analysis-and-use-of-museums-resources-by-classroom-educators-and-students/> 
2 "smithsonian-digital-learning - Teacher Toolkit (Research Findings)." 2013. 28 Sep. 2016 


<https://smithsonian-digital-learning.wikispaces.com/Teacher+Toolkit+(Research+Findings)> 
3 Kirschner, P. A. 2015. "Do We Need Teachers as Designers of Technology-Enhanced Learning?” 


<http://link.springer.com/article/10.1007/s11251 -015-9346-9> 


Methodology 


Using anonymous usage data from the SLL, a series of user actions was analyzed for two 
groups. Youth users (n=1452) in the project’s target age range of 13-18 were compared to 
adult users, anyone over 18 (n=7937) for a thirteen month time period spanning 2015 and 2016. 


Youth Users Adult Users 

13 - 17 (n=1452) 18+ (n=7937) 
Number of uploaded resources = 1109 6420 
Number of collections copied = 1021 5858 


The actions analyzed ranged in their level of complexity. Some actions require a single click 
while others require a series of steps and generation of descriptive data. We developed a 
simple scale to provide a comparative index of the complexity of various actions. Using a scale 
of 1-5, we assigned each action a “complexity score” relative to the steps required to fully 
execute the action: 


1 = single click to execute action 

2 = single click + addition of descriptive data to execute action 

3 = multiple clicks to execute action 

4 = multiple clicks + addition of descriptive data to execute action 

5 = single/multiple clicks + addition of descriptive and learning performance data to 
execute action 


| = Description of Process Complexity Score 


Favoriting an Item 


Copy a Collection 


Upload a Resource 


Create a Collection 


Start an Assignment 


Complete an Assignment 


Single click on favoriting icon appended to any 
resource, collection, or person 


Single click and the option to add a single piece of 
descriptive data for a new, personalize title 


Multiple clicks and addition of descriptive data to 
describe resource. 


Multiple clicks and addition of descriptive data to 
describe collection 


Multiple clicks and addition of descriptive data and 
learning performance data 


Multiple clicks and addition of descriptive data and 
learning performance data 


Findings 


Each chart presents the percentage of users that engaged in each of the analyzed actions. In 
the charts, X represents a combined range of individual actions conducted by less than 5% of 
the users. This allowed us to include a placeholder for low percentage outlier data* while 
showing dominant trends in user behavior comparing Youth Users (13-17 years of age) with 
Adult Users (18+ years of age). 


17% 


Of the 101 Youth Users that have favorited at least 1 | Of the 1,333 Adult Users that have favorited at least 
item in the Learning Lab, the top 3 results are below: | 1 item within the Learning Lab, the top 3 results are 
below: 


e 1 item favorited = 54 users (566%) e 1 item favorited = 566 users (43%) 
e 2 items favorited = 23 users (24%) e 2 items favorited = 224 users (17%) 
e 4items favorited = 7 users (7%) e 3 items favorited = 113 users (9%) 


4 As a means to highlight trends across data series for actions, any actions representing less than 5% of 
the user population is excluded. 


Number of Collections Copied by Users - Complexity Score = 2 


Of the 767 Youth Users that have copied at least 1 
collection in the Learning Lab, the top 3 results are 
below: 


e 1 collection copied = 623 users (81%) 
e 2collections copied = 94 users (12%) 
e 3-collections copied = 24 users (3%) 


Of the 2,786 Adult Users that have copied at least 1 
collection within the Learning Lab, the top 3 results 
are below: 


e 1 collection copied = 1760 users (63%) 
e 2collections copied = 540 users (19%) 
e 3collections copied = 195 users (7%) 


Number of Resources Uploaded by Users - Complexity Score = 3 


39% 


14% 


Of the 171 Youth Users that have uploaded at least | Of the 606 Adult Users that have uploaded at least 1 
1 resource within the Learning Lab, the top 3 results | resource within the Learning Lab, the top 3 results 
are below: are below: 


5 resources uploaded = 67 users (39%) e 1 resource uploaded = 142 users (26%) 
6 resources uploaded = 24 users (14%) e 2 resources uploaded = 68 users (13%) 
7 resources uploaded = 14 users (8%) e 3 resources uploaded = 55 users (10%) 


Number of Collections Created by Users - Complexity Score = 4 


Of the 116 Youth Users that have created at least 1 
collection within the Learning Lab, the top 3 results 
are below: 


e 1 collection created = 103 users (89%) 
e 2collections created = 9 users (8%) 
e 4Acollections created = 2 users (1%) 


Of the 519 Adult Users that have created at least 1 
collection within the Learning Lab, the top 3 results 
are below: 


e 1 collection created = 354 users (68%) 
e 2collections created = 84 users (16%) 
e 3-collections created = 21 users (4%) 
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Number of Assignments Started by Users - Complexity Score = 5 


Of the 477 Youth Users that have started at least 1 Of the 282 Adult Users that have started at least 1 
assignment within the Learning Lab, the top 3 results | assignment within the Learning Lab, the top 3 
are below: results are below: 


e 1 assignment started = 405 users (85%) e 1 assignment started = 204 users (72%) 
e 2 assignments started = 60 users (13%) e 2 assignments started = 65 users (23%) 
e 4 assignments started = 6 users (1%) e 3 assignments started = 4 users (2%) 
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Number of Assignments Completed by Users - Complexity Score = 5 


93% 
100% 


Of the 37 Youth Users that have completed at least | Of the 85 Adult Users that have completed at least 1 


1 assignment within the Learning Lab, the only result | assignment within the Learning Lab, the top 3 
recorded thus far is 1 assignment: results are below: 


e 1 assignment completed = 37 users (100%) 


1 assignments completed = 80 users (93%) 
2 assignments completed = 3 users (4%) 
3 assignments completed = 2 users (2%) 
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Summary 


The single highest number of actions executed by users between 13-17 yrs. old within the 
Smithsonian Learning Lab was uploading resources (1109 actions). 


It is worth noting, that actions taken by some youth use numbers may be influenced by 
directions provided by an educator. Many observations of teachers during this first year found 
them directing students to copy collections (1021 actions), and begin assignments (569) as a 
regular part of class-based activity. 


Considering that fact, actions such as favoriting an item (244) or developing a collection 
(139) have most likely been performed by youth users in ways largely unmitigated by an 
educator. Therefore, while their numbers are lower, they could be seen as viable actions to 
consider for future development given the likelinood that students largely elected to perform 
these actions. 


When comparing the trends present for youth users between 13-17 yrs. old against the same 
data for users 18+ yrs. old, all trends remain fairly consistent across the two populations with 
the following two exceptions: 


e Youth Users tend to use all of the tools slightly more than their adult counterparts. 
Across each action recorded, they have a slightly higher percentage of performing every 
one of the measured actions. 

e Youth Users uploaded more resources on average. They upload between 5-7 
resources on average whereas Adult Users tend to upload 1-3 resources. 


This analysis and report prepared by Navigation North Learning Solutions 
under the direction of The Smithsonian Center for Learning and Digital Access 
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